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ABSTRACT 

Fifth and eighth grade students in New York City 
participated in a citywide science survey for the fourth consecutive 
year in 1987. These surveys were designed to assess overall 
achievement in the pre-high school grades and to provide school, 
district and citywide summary information about student performance 
in basic areas of the science curriculum. Another purpose was to 
facilitate staff development in areas of greatest instructional need. 
The content areas included were biology, earth science, physics, 
chemistry, and process skills, plus geology, weather, and astronomy 
in grade eight. This document contains discussions of the process, 
administration, findings, reliability and validity of the test. 
Tables include: (1) percentages of students reaching mastery in 
general science; (2) citywide results by subtest and total test in 
both general and special education; (3) summary of students tested 
citywide; and (4) mean p-value by district. (CW) 
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SUMMARY OF THE CITYWIDE RESULTS OF THE 
NEW YORK CITY 1987 SCIENCE SURVEY 



GRADES 5 AND 8 

Students in grades 5 and 8 took citywide science surveys in 
May 1987 for the fourth consecutive year. These surveys are 
designed to assess the overall level of science achievemt.xt in 
the pre-high school grades and to provide school, district and 
citywide summary information about student performance in basic 
areas of the science curriculum. The purpose for collecting 
these data is to facilitate staff development in areas of 
greatest instructional need. 

In grade 5 students have shown steady improvement in science 
from 1985 to 1987 as measured by the percent of students meeting 
the mastery criterion. The content areas included were biology, 
earth science, phys^-rs, chemistry and process skills. 

In grade 8 a new earth science curriculum was introduced 
citywide in the 1986-87 year. The areas included in the new 
curriculum were geology, weather and astronomy. Because this was 
the first year of the new curriculum and the new content in the 
survey measuring it, no meaningful comparison with 1985 or 1986 
performance is possible. The survey on the whole proved to be 
difficult which may have resulted in part from unfamiliarity with 
the new curriculum. 



Grade 5^ 



1985 1986 1987 
General Science 



% of Students Reaching Mastery 49.8 51.5 ^^'^w 

% Items Correct for Mastery 65.0^ 58.0^ 65.0^ 

Mean % Items Correct 63.5 57.9 67.8 

Grade 8 

General Earth 

Science Science 

% Students Reaching Mastery 38.3 41.9 NA^ 

% Items Correct for Mastery 65.0^ 65.0^ NA^ 

Mean % Items Correct 58.2 60.1 47.5 



^ Excludes students in District 10 because of a breach in test 
security. 

^ Represent comparable levels of performance as determined by 
test equating studies • 

^ No mastery criterion was set because this was the first year of 
the new citywide earth science curriculum and survey. 
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DISCUSSION OF PROCESS AND RESULTS 

The Board of Education believes that testing and learning 
are integral parts of the same curriculum and instructional 
effort. Tests must validly reflect curriculum in order that 
student achievement of that curriculum be adequately measured and 
areas of strength and deficiency identified. Such identification 
is necessary to target instructional needs so that appropriate* 
action can be taken. Because no published test adequately re- 
flected objectives in the New York City and New York State 
science curricula, customized tests were developed in-house by 
the Division of Curriculum and Instruction in cooperation with 
the Office of Educational Assessment. Grades 5 and 8 were chosen 
as the target grades because they were considered to be crucial 
points in the sequence of science instruction. It is particular- 
ly important to identify possible problem areas in science 
instruction at these grades because problem areas not rentadiated 
at these points carry over into the next school administrative 
organizational level: intermediate or junior high and high 
school. This can seriously interfere with the instructional pro- 
gram at the succeeding level. 

The tests were designed primarily to provide group data in 
order to pinpoint schools or districts that may have 
instructional or s':aff develofxnent needs in their science 
programs. Although 1987 individual student scores were reported 
to schools and districts, there was no intention that such scores 
be reported to the individual students. 
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Because it was not possible to measure all the science 
objectives with one test, an advisory conunittee consisting of 
representatives from the community school districts was asked to 
identify and select appropriate important objectives for 
inclusion in the test at each grade. Items were then written to 
measure each objective. All items were reviewed for readability 
and vocabulary level, lack of ambiguity, elimination of possible 
bias, and to ensure that the item measured the objective and had 
one and only one correct answer . This procedure was followed in 
constructing the surveys at grades 5 and 8. 

The Spring 1987 grade 5 survey consisted of 60 items from 
the 1985 test. The grade 8 earth science survey consisted of 60 
items; some were taken from the 1986 earth science pilot test, 
some were revised pilot test items and ethers were developed 
specifically for the 1987 survey. 

TEST ADMINISTRATION 

On May 19, 1987 the grade 5 and 8 New York Citywide Science 
Surveys were administered. At grade 5, 56,772 students took the 
survey. (This excludes about 3,000 students in District 10 where 
a security breach was considered to have rendered test results 
invalid.) Of the 56,772 students tested, 4,739 were Special 
Education students. Adminiu^tration of the English version 
included 1,644 limited-English-proficient students while T,0S9 
students took the Spanish version. At grade 8, 51,27j students 
took the new earth science survey. Of these tested students, 
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4,208 v;ere from Special Education. Administration of the English 
version included 693 limited-English-proficient students while 
658 students took the Spanish version. 

FINDINGS 

The results of the 1987 science survey administration are re- 
ported in Tables 1 through 3. Tables lA, IB and 2 report overall 
citywide results for grades 5 and 8^ while Tables 3A and 3B report 
results by district. Tables lA and IB present comparisons of 1987 
with 1986 data whereas Tables 2, 3A and 3B present data for 1987 
only. 

Table lA reports 1987 performance for general education stu- 
dents in grades 5 and 8. In grade 5 in 1987 students performed 
better on the science survey than they had in 1986. In 1987^ 59* 8 
percent of the students met or exceeded the mastery criterion com- 
pared with 51.5 in 1986. In 1987 the grade 5 citj^ide average of 
67.8 percent correct of che 60 items was above the criterion level 
of 65 percent correct. This also is an indication of improved per- 
formance over 1986 where the citywide average of 57.9 percent cor- 
rect of the 50 items was right at the criterion level of 58 percent 
correct^. It is interesting tr^ note that as shown in Table 3A in 
grade 5 th^ 1987 average percent of items correct met or exceeded 
the criterion in 20 districts as compared with 17 districts in 1986. 

^ It should be noted that the grade 5 mastery criterion changed 
from 58 percent correct in 1986 to 65 percent correct ir 1987 
because different tests (with somewhat different levels of 
difficulty) were used in the two years. The two tests were 
equated to adjust for the difference in item difficulty, using 
Rasch procedures. 
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Because of the extensive nature of the change in the grade 8 
science curriculum and consequently in test content, it was not 
possible to make any comparisons between performance in 1986 and 
1987. Also because of these changes it was considered inappro- 
priate to estimate in advance a mastery criterion. Therefore, 
none was set. However, of the 60 items in the 1987 grade 8 
survey, the average was 47.5 percent of the items correct. 

It is of interest to lock at the subtest data for each 
grade. As shown in Table lA, in grade 5, except for chemistry 
where general education students had an average of 57 percent 
correct, the citywide performance on the other four subtests 
ranged from 68 to 71 percent of the items correct. This poor 
performance in chemistry relative to the other areas was also 
true of every district in grade 5 both in 1987 and 3 986. As 
shown in Table lA, grade 8 students had 50 percent of the items 
correct in the area of weather which was somewhat better than the 
46 and 45 percent correct in geology and astronomy, respectively. 

Citywide, boys performed somewhat better on the surveys than 
did girls. In grade 5 boys got 70 percent of the items correct 
and girls got 67 percent correct, while in grade 8 the percent of 
items correct was 50 for boys and 46 for girls. The New York 
City sex differences of 3 and 4 percentage points in grades 5 and 
8, respectively, are somewhat less than the sex differences of 
6 percentage points in grades 5 and 9 as reported for the 1983 
Second International Science Study. ^ 

^ The Newsletter of Teachers College/Columbia University . Spring 
1985 Volu-tie 13 Number 2. Office of Public Relations. Teachers 
College/Columbia University New York 10027. 
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RELIABILITY AND VALIDITY 

The reliability and validity of the New York City Science 
Surveys were assessed. The 1987 test was a reliable measure of 
science achievement at each grade, as indicated by reliability 
coefficients^ for the total test of .90 at grade 5 and .84 at 
grade 8. These figures indicate that the tests are homogeneous, 
and that the tests produce consistent results. 

Concent validi<-y, how well the content of the test is match- 
ed to curriculum content, is crucial to a measure of achievement 
such as the science survey. This was ensured by test develop- 
ment procedures at each grade. Objectives relevant to the 
science curriculum were identified and items constructed to 
measure these objectives. After item-editing to ensure appro- 
priate measurement of each objective, items were pilot tested. 
Analyses of the item data were conducted and appropriate items 
were selected for inclusion in the survey at each grade. Final 
test item selection was based upon content mix that reflected the 
curriculum and desirable psychometric characteristics of the items. 

CONCLUSIONS 

The 1987 science survey yielded useful information 
concerning the current level of science achievement in grades 5 
and 8. First and most important is that the results show that 
science achievement in grade 5 has improved over that of 1986. 
This improvement in rerlected in the increase in the percent of 



^ The Kuder-Richardson Formula 20 was used to measure test 
reliability. 
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students reaching the mastery criterion: 51.5 percent in 1986 and 
59.8 percent in 1987. 

The survey was designed to pinpoint areas of school and 
district strength and deficiency among the areas that constitute 
the basis of the science curriculum. In grade 5 these areas 
consisted of biology, earth science, physics, chemistry, and 
process sk.lls. The results showed that cltywide and by district 
performance was poorest in chemistry. Because the objectives to 
be measured and the items measuring these objectives had been 
judged to be of appropriate difficulty for the students to be 
tested, the relatively poor pertormance in chemistry appears to 
result from lack of student knowledge in this area. Results of 
the 1987 science survey at grade 5 were used to plan instructional 
and staff development activities. 

At grade 8, however, because of the new curriculum and test 
content, it was impossible to make any meaningful comparison of 
student perforniance in 1987 with that of previous years. The 1987 
survey proved to be difficult with students getting an average of 
only 48 percent of the 60 items correct. As was true at grade 5 
the objectives measured and the items measuring these objectives 
had been judged to be of appropriate difficulty for grade 8 students. 
While this relatively low level of performance may have resulted ir 
part from unfamiliarity with the new curriculum, it also suggested 
that the objectives and items be reviewea. The results of the 1987 
science survey at grade 8 were used to plan instructional and staff 
development activities. 
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TABLi; lA 

New York Citywide Science Surveys: 1987 
Citywide Results by Subtest and Total: Grades 5 and 8 

General Education Results 

% Total 





No. or 


Raw 


Score 


Items 




Test 


Items 


Mean 


S.D. 


Correct 


KR20 






Grade 5 - N 


= 52,033^ 






Biology 


9 


6.25 


1.86 


69.5 


.54 


Earth Science 


15 


10.34 


2.86 


69.0 


.69 


Physics 


18 


12.83 


3.25 


71.3 


.71 


Chemistry 


9 


5.15 


2.23 


57.2 


.65 


Process Skills 


9 


6.12 


2.21 


68.0 


.70 


Total 


60 


40.69 


in. 08 


67.8 


.90 


59,8 % of students 


reached criterion (65% of 


items correct) 





Grade 8 - N = 47,067 

Geology 15 6.89 2.51 45.9 .51 

weather 29 14.39 4.63 49.6 .71 

Astronomy 16 7.23 3.27 45.2 .69 

Total 60 28.51 8.87 47.5 .84 



Excludes students in District 10. Because of a breach in 
security District 10 test results for grade 5 were considered 
invalid. 
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TABLE IB 

New York Citywide Science Surveys: 1987 
Citywide Results by Subtest and Total. Grades 5 and 8 

Special Education Results 

% Total 



No. of Raw Score Items 

Test Items Mean S.D. Correct KR20 

Grade 5 - N = 4,739* 

Biology 9 4.20 1.98 46.7 .49 

Earth Science 15 6.50 2.95 43.3 .64 

Physics 18 8.04 3.81 44.7 .74 

Chemistry 9 2.97 1.86 33.0 .48 

Process Skills 9 3.54 2.12 39.4 .60 

Total 60 25.25 9.93 42.1 .87 



11.2 % of students tsac" 2d criterion (65% of items correct) 







Grade 8 - 


N = 4,208 




Geology 


15 


4.94 


2.10 


32.9 


Weather 


29 


9.86 


3.71 


34.0 


Astronomy 


16 


4.51 


2.39 


28.2 


Total 


60 


19.31 


6.Z4 


32.2 



Excludes students in District 10. Because of a breach in 
security District 10 test results for grade 5 were considered 
invalid. 
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TABLE 2 

New York City Science Survey 1987 
Summary of Students Tested Citywide 
Grades 5 and 8 





Grade 5* 




Grade 8 






General 


Special 




General 


Special 






Education 


Education 


Total 


Education 


Education 


Total 


Answer Sheet 














Received 














rested^ 


52,033 


4,739 


56,112 


47,067 


4,208 


51,275 


Absent 


3,533 


732 


4, 265 


6,671 


1,342 


8,013 


Exempt LEP 


1.670 


24 


1,694 


1,270 


18 


1, 288 


Exempt SE 


0 


179 


179 


0 


108 


108 


Excused 


11 


6 


17 


4 


3 


mm 

1 


No Score 














(no reason) 


270 


99 


«^ /- 
369 


904 


105 


1, 009 


TOTAL 


57,517 


5,779 


63,296 


r5, 916 


5,784 


61,700 


Language Group 














English Version 












English Prof. 


49,418 


4,0^1 


54,039 


43,832 


4,052 


49,884 


Lim. Eng. Prof. 


1,556 


88 


1 f^AA 


566 


127 




TOTAL 


50,974 


4,709 


55,683 


46,398 


4,179 


50,577 


Spanish Version 1,059 


30 


1,089 


669 


29 


698 


TOTAL 


52,033 


4,739 


56,772 


47,067 


4,208 


51,275 


Sex 














Males 


28,293 


4,086 


32,379 


27,536 


4,056 


31,592 


Females 


28,950 


1,648 


30,598 


28,053 


1,694 


29,747 


Miscodes 


274 


45 


319 


327 


34 


361 


TOTAL 


57,517 


5,779 


63,296 


55,916 


5,784 


61,700 



Excludes students in District 10. Because of a breach in security. 
District 10 results for grade 5 were considered invalid. 

Basis for school, district, citywide reports. 



TABLE 3B 

New York Citywide Science Survey 1987 
Mean P-Value^ by District 

Grade 8 - General Education 

Total 

Subtests Test N 

Geology Weather Astronomy 



No. of 





15 

X ^ 


29 


16 


60 


















41 

^ X 


45 


42 


43 


781 


2 


51 

^ X 


56 


52 


54 


1, 224 


w 


42 


44 


40 


43 


740 


4 


43 


45 


40 


43 


992 


5 


43 


45 


38 


42 


690 


6 


41 


44 


39 


42 


1,592 


7 


41 

"t X 


4^ 


40 


42 


1,001 


fi 

o 


45 


48 


43 


46 


1, 563 


9 


40 


43 


37 


41 


1,745 


10 


43 


47 


42 


45 


2,468 


11 
X X 


56 


60 


55 


58 


1,773 


12 

X *• 


40 


44 


38 


41 


749 


13 

X W 


47 


46 


42 


45 


1,191 


14 

X "1 


41 

"t X 


46 


43 


44 


1, 258 


15 


43 


47 


43 


45 


1,138 


16 


40 


44 


36 


41 


628 


17 


41 


44 


38 


42 


2,274 


18 


50 


52 


49 


51 


1,419 


19 


43 


47 


43 


45 


1,674 


20 


47 


51 


47 


49 


2,075 


21 


49 


53 


51 


52 


1,723 


22 


51 


55 


51 


53 


1,669 


23 


41 


42 


35 


40 


938 


24 


49 


53 


49 


51 


2,161 


25 


53 


57 


56 


56 


1,610 


26 


45 


54 


48 


50 


616 


27 


46 


50 


45 


48 


2,227 


28 


51 


55 


52 


53 


1,378 


29 


45 


49 


45 


47 


1,859 


30 


45 


51 


48 


49 


1,531 


31 


49 


54 


51 


52 


2,873 


32 


44 


49 


43 


46 


1,176 


33 


55 


59 


57 


58 


319 


78 


37 


43 


34 


39 


11 


Total 


46 


50 


45 


48 


47,067 


* Mean 


p-value is 


the percent of 


items 


answered 


correctly. 



TABLE 3 A 

New York Citywide Science Survey 1987 
Mean P-Value^ by District 



Grade 5 - General Education 



Total 

Subtests Test N 



Bi^. E . Sc . Phys . Chem . Pr . Sk. 

No. of 

Items 9 15 18 9 9 60 



District 



1 


0 H 






0 0 




0 ^ 




Q C>l 




1 A 


/ J 




7 


61 


7*5 


77 
/ ^ 


1 f 4o / 


-a 




O J 




O O 








Q c: A 


A 


O 4 


O J 




o / 




0 u 




1 f UOU 






Al 






Al 




61 


7 ^ J 










D 4 


AVi 
40 


6n 


o u 


9 1 Lt\ 
C f 14v 


n 




61 
o J 




67 


Al 
4 / 


60 


69 
o c 


1 191 
1 f 1 ^ J 


o 


70 


6R 




6R 


^ o 


61 


r.6 

wO 


1 6ft9 


Q 








6^ 


Al 
4 / 




6n 
o u 


9 9Q9 


1 0 


















1 1 
J- X 


71 


77 




76 


6^ 

O 4 


in 


71 


1 flftl 
1 f oo J 




61 


61 




66 


4 i7 


CO 


61 

0 1 


1 f J 3 V 




68 
u o 


6Q 




71 


S7 


6R 
c o 


6fl 
oo 


1 ^sn 

1 f 4 0 V 


14 


70 


68 




71 


54 


66 


67 


1,533 


15 


68 


67 




71 


56 


68 


67 


1,749 


16 


71 


69 




70 


56 


66 


67 


974 


17 


64 


65 




68 


53 


63 


64 


2,645 


18 


72 


75 




77 


63 


74 


73 


1,598 


19 


64 


64 




68 


49 


62 


63 


2,224 


20 


73 


75 




76 


67 


74 


74 


1,861 


21 


70 


71 




73 


60 


71 


70 


1,515 


22 


76 


74 




11 


64 


74 


74 


2,214 


23 


64 


63 




67 


52 


62 


63 


1,120 


24 


71 


72 




73 


59 


71 


70 


2,127 


25 


78 


76 




11 


67 


78 


76 


1,745 


26 


80 


81 




81 


72 


81 


79 


1,072 


21 


71 


69 




72 


58 


70 


69 


2,444 


28 


71 


71 




71. 


61 


71 


70 


1,671 


29 


72 


71 




73 


61 


70 


70 


1,980 


30 


71 


74 




76 


62 


72 


72 


1,884 


31 




74 




75 


66 


77 


74 


2,999 


32 


62 


61 




64 


49 


60 


60 


1,397 


33 


73 


75 




81 


74 


81 


78 


83 


Total 


70 


69 




71 


57 


68 


68 


52,033 


a 


Mean p- value is 


the 


percent 


Of 


items answered correctly. 


b 


Because of 


a breach 


in test 


security, test 


results 


for 




District 10 


were 


considered 


to 


be invalid. 
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